Emergent Dynamics of Joy, Distress, Hope and Fear in Reinforcement Learning Agents
نویسندگان
چکیده
We report on a study that shows plausible emotion dynamics for joy, distress, hope and fear, emerging in an adaptive agent that uses Reinforcement Learning (RL) to adapt to a task. Joy/distress is a signal that is derived from the RL update signal, while hope/fear is derived from the utility of the current state. Agent-based simulation experiments replicate psychological and behavioral dynamics of emotion including: joy and distress reactions that develop prior to hope and fear; fear extinction; habituation of joy; and, task randomness that increases the intensity of joy and distress. This work distinguishes itself by assessing the dynamics of emotion in an adaptive agent framework coupling it to the literature on habituation, development, and extinction. Our results support the idea that the function of emotion is to provide a complex feedback signal for an organism to adapt its behavior. We show this feedback signal can be operationalized for RL agents. This is important because (a) RL-based models can help understand the relation between emotion and adaptation in animals, (b) the emotional state might be used to increase adaptive potential, and (c) expression of an emotion to a human observer that it is grounded in the learning mechanism of the agent should help interpret the meaning of the emotion.
منابع مشابه
Joy, distress, hope, and fear in reinforcement learning
In this paper we present a mapping between joy, distress, hope and fear, and Reinforcement Learning primitives. Joy / distress is a signal that is derived from the RL update signal, while hope/fear is derived from the utility of the current state. Agent-based simulation experiments replicate psychological and behavioral dynamics of emotion including: joy and distress reactions that develop prio...
متن کاملA reinforcement learning model of joy, distress, hope and fear
In this paper we computationally study the relation between adaptive behavior and emotion. Using the Reinforcement Learning framework, we propose that learned state utility, V(s), models fear (negative) and hope (positive) based on the fact that both signals are about anticipation of loss or gain. Further, we propose that joy/distress is a signal similar to the error signal. We present agent-ba...
متن کاملFear and Hope Emerge from Anticipation in Model-Based Reinforcement Learning
Social agents and robots will require both learning and emotional capabilities to successfully enter society. This paper connects both challenges, by studying models of emotion generation in sequential decision-making agents. Previous work in this field has focussed on model-free reinforcement learning (RL). However, important emotions like hope and fear need anticipation, which requires a mode...
متن کاملCombating Reinforcement Learning's Sisyphean Curse with Intrinsic Fear
To use deep reinforcement learning in the wild, we might hope for an agent that would never make catastrophic mistakes. At the very least, we could hope that an agent would eventually learn to avoid old mistakes. Unfortunately, even in simple environments, modern deep reinforcement learning techniques are doomed by a Sisyphean curse. Owing to the use of function approximation, these agents even...
متن کاملHierarchical Functional Concepts for Knowledge Transfer among Reinforcement Learning Agents
This article introduces the notions of functional space and concept as a way of knowledge representation and abstraction for Reinforcement Learning agents. These definitions are used as a tool of knowledge transfer among agents. The agents are assumed to be heterogeneous; they have different state spaces but share a same dynamic, reward and action space. In other words, the agents are assumed t...
متن کامل